Display control system, display control method, computer-readable storage medium having stored thereon display control program, and display control apparatus

ABSTRACT

While an example terminal device is being held with its long sides extending horizontally, when a left-right direction is inputted by using a right analog stick, a first virtual camera is directed to a left-right direction in a virtual space. On the other hand, while the terminal device is being held with its long side extending vertically, when an apparent left-right direction, that is, an up-down direction in the state where the terminal device is held with its long sides extending horizontally, is inputted by using a right analog stick, the first virtual camera is directed to the left-right direction in the virtual space.

CROSS REFERENCE TO RELATED APPLICATION

The disclosure of Japanese Patent Application No. 2012-085098, filed onApr. 4, 2012, is incorporated herein by reference.

FIELD

The technology disclosed herein relates to a display control system, adisplay control method, a computer-readable storage medium having storedthereon a display control program, and a display control apparatus, forcausing a display section to display an image in accordance with anoperation performed by a user.

BACKGROUND AND SUMMARY

Conventionally, there is a display device that changes a displaycontent, using the attitude of an apparatus. For example, there is anapparatus that detects the gravity direction, rotates an object based onthe gravity direction, and causes a display device to display therotated object.

However, the conventional apparatus rotates the object in accordancewith the attitude of the apparatus and causes the rotated object to bedisplayed, but does not control a display image based on an inputoperation performed by a user.

Therefore, an object of an embodiment is to provide a technology thatcan control a display image by using an input operation performed onto adirection input section, while taking the attitude of the apparatus intoconsideration.

An embodiment has adopted the following configurations to solve theabove issue.

One embodiment is a display control system which controls, in accordancewith an operation performed onto an operation device which includes adirection input section and a display section, a display content for thedisplay section. The display control system includes an input dataobtaining section, a first virtual camera control section, and a displaycontrol section. The input data obtaining section obtains input data inaccordance with an input direction inputted by using the direction inputsection. The first virtual camera control section controls a virtualcamera in a virtual space based on the input direction. The displaycontrol section causes the display section to display an image based onthe virtual camera controlled by the first virtual camera controlsection. The first virtual camera control section changes the inputdirection indicated by the input data, in accordance with a change in anattitude of the operation device, and controls the virtual camera basedon the changed direction.

According to the above, it is possible to control the virtual camera byan input direction inputted by using the direction input section of theoperation device, and to change the input direction in accordance withthe attitude of the operation device.

Further, in another configuration, the first virtual camera controlsection may control the virtual camera based on an input directioninputted by using the direction input section, the input direction beingbased on a real space.

According to the above, it is possible to control the virtual camerabased on an input direction inputted by using the direction inputsection, the input direction being based on the real space, irrespectiveof the attitude of the operation device. For example, even when theattitude of the operation device has been changed, it is possible tocontrol the virtual camera based on an input in an apparent rightdirection.

Further in another configuration, the display control system may furtherinclude a second virtual camera control section which controls anorientation of the virtual camera, in accordance with the attitude ofthe operation device.

According to the above, it is possible to further control theorientation of the virtual camera in accordance with the attitude of theoperation device.

Further, in another configuration, the first virtual camera controlsection may change an orientation of the virtual camera such that thechanged direction of the orientation of the virtual camera agrees with adirection in the virtual space corresponding to an input directioninputted by using the direction input section, the input direction beingbased on a real space.

According to the above, even when the attitude of the operation devicehas been changed, it is possible to change the orientation of thevirtual camera such that the input direction inputted by using thedirection input section, the input direction being based on the realspace, agrees with the changed direction of the orientation of thevirtual camera. Accordingly, for example, when the input direction(apparent input direction) inputted by using the direction inputsection, the input direction being based on the real space, is the rightdirection, it is possible to direct the orientation of the virtualcamera to the right direction.

Further, in another configuration, the first virtual camera controlsection may rotate the input direction in accordance with a rotation ofthe operation device about an axis perpendicular to a screen of thedisplay section, and control an orientation of the virtual camera basedon the rotated input direction.

According to the above, when the operation device is rotated about anaxis perpendicular to the screen of the display section, it is possibleto rotate the input direction inputted by using the direction inputsection in accordance with the rotation of the operation device, and tocontrol the orientation of the virtual camera based on the rotated inputdirection.

Further, in another configuration, based on a component in apredetermined direction of the input direction, the first virtual cameracontrol section may change an orientation of the virtual camera into apredetermined direction in the virtual space.

According to the above, it is possible to extract a component in apredetermined direction (e.g., left-right direction) of the inputdirection, and to change the orientation of the virtual camera into thepredetermined direction (e.g., left-right direction) in the virtualspace.

Further, in another configuration, in accordance with a rotation of theoperation device about an axis perpendicular to a screen of the displaysection, the second virtual camera control section may rotate thevirtual camera about an imaging axis thereof in the rotation directionof the operation device.

According to the above, when the operation device is rotated about anaxis perpendicular to the screen of the display section, it is possibleto rotate an image displayed on the display section in a reversedirection. Accordingly, it is possible to prevent the image displayed onthe display section from being apparently rotated.

Further, in another configuration, the display control system mayfurther include a user object control section configured to control amoving direction of a user object movable in the virtual space, based onan input direction inputted by using the direction input section or asecond direction input section provided in the operation device. Theuser object control section changes the input direction in accordancewith the change in the attitude of the operation device, and controlsthe moving direction of the user object based on the changed direction.The display control section causes the display section to display animage including the user object.

According to the above, it is possible to move the user object in thevirtual space, based on the input direction inputted by using thedirection input section.

Further, in another configuration, the display control system mayinclude a third virtual camera control section and a second displaycontrol section. The third virtual camera control section changes anorientation of a second virtual camera arranged in the virtual space, inaccordance with an orientation of the virtual camera. The second displaycontrol section causes a display device other than the display sectionto display an image of the virtual space viewed from the second virtualcamera.

According to the above, it is possible to cause a display device otherthan the display section to display an image, using the second virtualcamera.

Further, another example is a display control system configured tocontrol, in accordance with an operation performed onto an operationdevice which includes a direction input section and a display section, adisplay content for the display section. The display control systemincludes an input data obtaining section, a control section, and adisplay control section. The input data obtaining section obtains inputdata in accordance with an input direction inputted by using thedirection input section. The control section controls a predeterminedobject in a virtual space based on the input direction. The displaycontrol section causes the display section to display an image inaccordance with an attitude of the operation device and causes thedisplay section to display an image based on a predetermined objectcontrolled by the control section. Further, the control section changesthe input direction indicated by the input data in accordance with achange in the attitude of the operation device, and controls thepredetermined object based on the changed direction.

According to the above, it is possible to control a predetermined objectby an input direction inputted by using the direction input section ofthe operation device, and to change the input direction in accordancewith the attitude of the operation device. Here, the predeterminedobject may be, for example, a virtual camera or an object operated by auser.

Further, another embodiment may be a display control method performed inthe above display control system. Such a display control system may beconfigured by a single apparatus or a plurality of apparatuses. Further,another example may be a display control program which causes a computerto function as the sections included in the above-described displaycontrol system. Further, another example may be a display controlapparatus including the sections included in the above-described displaycontrol system.

According to the present embodiment, it is possible to control a displayimage by an input operation performed onto a direction input section, inaccordance with the attitude of the operation device.

These and other objects, features, aspects and advantages of theexemplary embodiment will become more apparent from the followingdetailed description when taken in conjunction with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view showing a non-limiting example of aconfiguration of a game system 1;

FIG. 2 is a block diagram showing a non-limiting example of aconfiguration of the game system 1;

FIG. 3 is a non-limiting example of images displayed on a display device2 and an LCD 71 of a terminal device 7 at the time when the game of thepresent embodiment is performed;

FIG. 4 illustrates a non-limiting example of a change in the orientationof a first virtual camera C1 at the time when the terminal device 7 isrotated in a yaw direction (about Y-axis) from a reference attitude;

FIG. 5 illustrates a non-limiting example of a change in the orientationof the first virtual camera C1 at the time when the terminal device 7 isrotated in a pitch direction (about X-axis) from the reference attitude;

FIG. 6 illustrates a non-limiting example of a change in the orientationof the first virtual camera C1 at the time when the terminal device 7 isrotated in a roll direction (about Z-axis) from the reference attitude;

FIG. 7 illustrates a non-limiting example of a change in the orientationof the first virtual camera C1 at the time when a right direction(X-axis negative direction) is inputted by using a right analog stick72B;

FIG. 8 illustrates a non-limiting example of a change in the orientationof the first virtual camera C1 at the time when a direction input isperformed by using the right analog stick 72B, while the terminal device7 is being held such that X-axis thereof is directed toward the gravitydirection (vertical holding);

FIG. 9 illustrates a non-limiting example of a change in the orientationof the first virtual camera C1 at the time when a direction input isperformed by using the right analog stick 72B, while the terminal device7 is being held (inclined holding) such that X-axis thereof is inclinedby a predetermined angle relative to the horizontal direction in thereal space;

FIG. 10 shows a non-limiting example of how an input vector inaccordance with an input direction of the right analog stick 72B isrotated in accordance with the attitude of the terminal device 7;

FIG. 11 shows a non-limiting example of a control of the orientation ofthe first virtual camera C1 based on the attitude of the terminal device7 and a direction input operation using the right analog stick 72B;

FIG. 12 illustrates a control of two virtual cameras based on anoperation of rotating the terminal device 7 in the yaw direction and aninput in the left-right direction using the right analog stick 72B;

FIG. 13 illustrates a non-limiting example of a control of two virtualcameras based on an operation of rotating the terminal device 7 in thepitch direction;

FIG. 14 illustrates a non-limiting example of a control of two virtualcameras based on an operation of rotating the terminal device 7 in theroll direction;

FIG. 15 shows a non-limiting example of images displayed on the terminaldevice 7 and the display device 2 when a position on the screen of thedisplay device 2 is indicated by using a controller 5;

FIG. 16 illustrates a non-limiting example of a position at which anindicated object 87 is located;

FIG. 17 shows a non-limiting example of how a user object 81 is movedbased on a direction input using a left analog stick 72A of the terminaldevice 7;

FIG. 18 illustrates a non-limiting example of a movement of the userobject 81 at the time when a direction input operation is performed byusing the left analog stick 72A while the terminal device 7 is beingheld vertically;

FIG. 19 shows a non-limiting example of a scene where the user object 81has moved and arrived at a house object 82 indicated by a second user;

FIG. 20 shows a non-limiting example of a scene where the user object 81has moved and arrived at a building object 89 arranged in the virtualspace;

FIG. 21 shows non-limiting various examples of positional relationshipsbetween the first virtual camera C1 and the second virtual camera C2 andimaging directions (sight line directions) thereof.

FIG. 22 shows a non-limiting example of an image displayed on thedisplay device 2 when the second virtual camera C2 is set at a viewpointC shown in FIG. 21;

FIG. 23 is a non-limiting example of an image displayed on the displaydevice 2 when the second virtual camera C2 is set at a viewpoint D shownin FIG. 22;

FIG. 24 shows a non-limiting example of various types of data stored ina game apparatus 3; and

FIG. 25 is a non-limiting example of a main flowchart showing the flowof processes performed in the game apparatus 3.

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS An Example ofConfiguration of Game System

Hereinafter, a game system 1 according to an embodiment will bedescribed with reference to the drawings. FIG. 1 is an external viewshowing a non-limiting example of a configuration of the game system 1.FIG. 2 is a block diagram showing a non-limiting example of aconfiguration of the game system 1. As shown in FIG. 1 and FIG. 2, thegame system 1 includes a stationary display device 2 (e.g., television),a stationary game apparatus 3, an optical disc 4, a controller 5, amarker device 6, and a terminal device 7.

The game apparatus 3 includes a CPU 10, a memory 11, and the like. Whena program stored in a storage medium such as the optical disc 4 isloaded onto a memory, the CPU 10 executes the program loaded onto thememory, thereby performing processing described later. Then, the gameapparatus 3 outputs an image as a result of the processing, to thedisplay device 2 and the terminal device 7.

The terminal device 7 is a portable display device which is small enoughto be held by both hands or one hand of a user. The user can move theterminal device 7 by holding it with his/her hand(s), or can locate theterminal device 7 at any position to use it. The terminal device 7includes an LCD (liquid crystal display) 71 which is a display device.The terminal device 7 also includes an input section 72 such as ananalog stick (left analog stick 72A, right analog stick 72B) forinputting a direction, an inertial sensor (gyro sensor, accelerometer,and the like) 73 capable of outputting attitude data in accordance withthe attitude of the terminal device 7, and the like. The terminal device7 and the game apparatus 3 are capable of wirelessly communicating witheach other. The terminal device 7 receives data of an image generated bythe game apparatus 3 from the game apparatus 3 and displays the image onthe LCD 71. Moreover, the terminal device 7 transmits, to the gameapparatus 3, operation data based on operations performed onto theterminal device 7 (operation onto the input section 72 and an operationto change the attitude of the terminal device 7).

The controller 5 is small enough to be held by a user, and is used toindicate a position on the screen of the display device 2. For example,by an imaging section provided in the controller 5 receiving infraredlight from the marker device 6 arranged at predetermined position on thedisplay device 2, the indicated position on the screen is detected.Alternatively, a sensor for detecting an attitude is provided in thecontroller 5, and a position on the screen may be indicated by theattitude of the controller 5 (for example, the middle point of thescreen is indicated for the initial attitude, and the indicated positionchanges in accordance with a change in the attitude from the initialattitude).

The terminal device 7 is held by a first user, and the controller 5 isheld by a second user, which will be described in detail later.

It should be noted that the game system 1 shown in FIG. 1 and FIG. 2 ismerely an example and the processing described later may be performed inany apparatus.

(Outline of Processing)

Next, an outline of the processing performed in the game system 1 of thepresent embodiment will be described. FIG. 3 shows a non-limitingexample of images displayed on the display device 2 and the LCD 71 ofthe terminal device 7 at the time when the game of the presentembodiment is performed. In the present embodiment, the terminal device7 is held by the first user, and the controller 5 is held by the seconduser.

As shown in FIG. 3, a user object 81, a house object (selection object)82, and the like, which are arranged in a virtual space defined by anxyz coordinate system, are displayed on the display device 2. Moreover,a rock object 83, a tree object 84 (84 a to 84 c), and the like arearranged in the virtual space. The user object 81 is a virtual characteroperated by the first user using the terminal device 7. An image similarto that on the display device 2 is displayed on the LCD 71 of theterminal device 7. As shown in FIG. 3, the image displayed on theterminal device 7 is an image whose angle of view is narrower than thatof the image displayed on the display device 2. The images displayed onthe terminal device 7 and the display device 2 are generated by using afirst virtual camera C1 and a second virtual camera C2 arranged in thevirtual space, respectively, and are changed in accordance with anoperation performed onto the terminal device 7.

Specifically, the orientations (attitudes) of the first virtual cameraC1 and the second virtual camera C2 are controlled by the attitude ofthe terminal device 7 and an operation performed onto the right analogstick 72B of the terminal device 7. For example, when the Y-axis (axisalong the up-down direction) of the XYZ coordinate system fixed for theterminal device 7 is parallel to the gravity direction, the X-axis (axisalong the left-right direction) is parallel to the ground, and theZ-axis (axis perpendicular to the LCD 71) is directed to the screen ofthe display device 2, the images shown in FIG. 3 are displayed on theterminal device 7 and the display device 2. When the attitude of theterminal device 7 is changed relative to this attitude of the terminaldevice 7 (hereinafter, this may be referred to as “reference attitude”),the orientations of the first virtual camera C1 and the second virtualcamera C2 are also changed. It should be noted that the attitude whenthe Y-axis is inclined by a predetermined angle relative to the groundin the real space may be defined as the reference attitude. When theuser sees the screen while holding the terminal device 7, it may beeasier to see the screen when the terminal device 7 is slightly inclinedthan when the terminal device 7 is held vertically (such that the Y-axisis parallel to the gravity direction). Therefore, such an attitude ofthe terminal device 7 may be defined as the reference attitude.

Hereinafter, control of the first virtual camera C1 by using operationsperformed onto the terminal device 7 (operation to change the attitudeof the terminal device 7 itself, and an direction input operationperformed onto the right analog stick 72B) will be described withreference to FIG. 4 to FIG. 10.

First, a change in the orientation of the first virtual camera C1 inaccordance with a change in the attitude of the terminal device 7 willbe described with reference to FIG. 4 to FIG. 6. FIG. 4 illustrates anon-limiting example of a change in the orientation of the first virtualcamera C1 at the time when the terminal device 7 is rotated in a yawdirection (about Y-axis) from the reference attitude. FIG. 5 illustratesa non-limiting example of a change in the orientation of the firstvirtual camera C1 at the time when the terminal device 7 is rotated in apitch direction (about X-axis) from the reference attitude. FIG. 6illustrates a non-limiting example of a change in the orientation of thefirst virtual camera C1 at the time when the terminal device 7 isrotated in a roll direction (about Z-axis) from the reference attitude.

In FIG. 4, a view of the terminal device 7 seen from above in the realspace is shown, and a view of the first virtual camera C1 seen fromabove the camera (from above in the virtual space) is shown. Forexample, as shown in FIG. 4, when the terminal device 7 is in thereference attitude, the imaging direction (Zc1-axis) of the firstvirtual camera C1 is directed to the rock object 83 which is arranged inthe virtual space, and the rock object 83 is displayed in the centralregion in the left-right direction on the screen of the terminal device7. From this state, when the rear surface of the terminal device 7 isdirected rightward in the real space, the first virtual camera C1 isalso directed rightward in the virtual space. Specifically, when theterminal device 7 is rotated in the yaw direction (about Y-axis), thefirst virtual camera C1 is rotated about Yc1-axis in an Xc1Yc1Zc1coordinate system fixed for the first virtual camera C1, in accordancewith the rotation angle. Here, Xc1-axis extends in the left directionrelative to the first virtual camera C1, and Yc1-axis extends in the updirection relative to the first virtual camera C1, and Zc1-axis extendsin the imaging direction of the first virtual camera C1. By the firstvirtual camera C1 rotating about Yc1-axis, the rock object 83, which hasbeen displayed in the central region of the screen of the terminaldevice 7 before the rotation of the terminal device 7, is moved to aleft region of the screen after the rotation of the terminal device 7.Moreover, the house object 82 having been displayed in a right region ofthe screen before the rotation is displayed near the center of thescreen after the rotation of the terminal device 7. Moreover, a treeobject 84 b having been displayed in a left region of the screen beforethe rotation comes outside the imaging range of the first virtual cameraC1 after the rotation, and is not displayed on the screen any more.

FIG. 5 shows the terminal device 7 viewed from the right direction andthe first virtual camera C1 viewed from the right of the camera at thattime. When the terminal device 7 is in the reference attitude as shownin FIG. 5, the imaging direction (Zc1-axis) of the first virtual cameraC1 is parallel to the ground (xz-plane) in the virtual space.Accordingly, the horizon in the virtual space is displayed on the screenof the terminal device 7, near the center in the up-down direction ofthe screen. When the rear surface of the terminal device 7 is directedupward in the real space from this state, the first virtual camera C1 isalso directed upward in the virtual space. Specifically, when theterminal device 7 is rotated in the pitch direction (about X-axis), thefirst virtual camera C1 is rotated about Xc1-axis in accordance with therotation angle. Accordingly, the rock object 83 and the like, which havebeen displayed in the central region in the up-down direction of thescreen of the terminal device 7 before the rotation of the terminaldevice 7, are displayed, after the rotation of the terminal device 7, ina lower region in the up-down direction of the screen. A sun object 85,which was not displayed on the screen before the rotation because it waslocated upper than and outside the screen in the virtual space, hasentered the imaging range of the first virtual camera C1 after therotation, and is displayed on the screen.

Moreover, FIG. 6 shows a view of the terminal device 7 viewed from thefront thereof, and a view of the first virtual camera C1 viewed from therear of the camera at that time. As shown in FIG. 6, when the terminaldevice 7 is in the reference attitude, the imaging direction (Zc1-axis)of the first virtual camera C1 is not rotated, and Xc1-axis fixed forthe first virtual camera C1 is parallel to the ground in the virtualspace. When the terminal device 7 is rotated in the roll direction(about Z-axis) from this state, the first virtual camera C1 is rotatedabout Zc1-axis in accordance with the rotation angle. Specifically, thefirst virtual camera C1 is rotated about Zc1-axis (imaging axis), in thesame rotation direction as the rotation direction about Z-axis of theterminal device 7, by the same rotation amount as the rotation amountabout Z-axis of the terminal device 7. Accordingly, the image generatedby using the first virtual camera C1 is rotated in a reverse directionof the rotation direction of the terminal device 7, about the centralaxis of the screen of the terminal device 7. As a result, the rotationin the roll direction of the terminal device 7 is canceled, and theimage is not apparently rotated in the roll direction on the screen ofthe terminal device 7 when viewed by the first user. In other words, theground (xz-plane) in the virtual space included in the image displayedon the screen of the terminal device 7 is parallel to the ground in thereal space, and the up-down direction in the virtual space is parallelto the gravity direction in the real space.

Therefore, for example, when the first user rotates the terminal device7 by 90 degrees counterclockwise in the roll direction, and thus X-axisis oriented toward the gravity direction (holding the terminal device 7in such an attitude may be referred to as “vertical holding”), the imageis rotated by 90 degrees clockwise, and the image having longer sides inthe up-down direction is displayed on the terminal device 7 (see FIG.8).

Rotations of the yaw direction, the pitch direction, and the rolldirection of the terminal device 7 can be calculated, based on aphysical quantity detected by the inertial sensor 73 of the terminaldevice 7, such as an angular velocity detected by a gyro sensor and/or agravitational acceleration detected by an accelerometer. For example, byintegrating by time the angular velocities about X, Y, and Z-axesdetected by a gyro sensor, the attitude of the terminal device 7(rotations about the respective axes) can be calculated. It should benoted that the detection of the attitude of the terminal device 7 is notnecessarily performed by the inertial sensor but other methods (forexample, an image of the terminal device 7 is taken by a camera, and theterminal device 7 is detected based on the taken image, and the like)may be employed.

Next, control of the orientation of the first virtual camera C1 based ona direction input using the right analog stick 72B will be described.FIG. 7 illustrates a non-limiting example of a change in the orientationof the first virtual camera C1 at the time when a right direction(X-axis negative direction) is inputted by using the right analog stick72B.

As shown in FIG. 7, when the right direction is inputted by using theright analog stick 72B (when the right analog stick 72B is slid in theright direction), the first virtual camera C1 is rotated in the yawdirection in accordance with the input amount (slide amount and slidetime period). Specifically, the first virtual camera C1 is rotated aboutan axis parallel to y-axis in the virtual space. Therefore, when theright direction is inputted using the right analog stick 72B, the firstvirtual camera C1 is directed rightward in the virtual space, and therock object 83 having been displayed near the center in the left-rightdirection of the screen before the input is now displayed in a leftregion in the left-right direction of the screen after the input. InFIG. 7, since y-axis fixed in the virtual space and Yc1-axis fixed forthe first virtual camera C1 are parallel to each other, the firstvirtual camera C1 is rotated about Yc1-axis.

When the left direction is inputted by using the right analog stick 72B,the first virtual camera C1 is reversely rotated about y-axis.Therefore, when the left direction is inputted by using the right analogstick 72B, the first virtual camera C1 is directed leftward in thevirtual space.

Further, in FIG. 7, when an upper right direction is inputted by usingthe right analog stick 72B, for example, only a component in the rightdirection of the input is used for rotating the first virtual camera C1in the yaw direction. Further, in FIG. 7, when an up-down direction isinputted by using the right analog stick 72B, the first virtual cameraC1 is not rotated in the yaw direction. That is, the first virtualcamera C1 is rotated in the yaw direction (left-right direction in thevirtual space) in accordance with an input in the left-right directionusing the right analog stick 72B, whereas the first virtual camera C1 isnot rotated in the yaw direction, the pitch direction (up-down directionin the virtual space), or the roll direction, for an input in theup-down direction using the right analog stick 72B.

FIG. 8 illustrates a non-limiting example of a change in the orientationof the first virtual camera C1 at the time when a direction input isperformed by using the right analog stick 72B, while the terminal device7 is being held such that X-axis thereof is oriented toward the gravitydirection (vertical holding). As shown in FIG. 8, in the case where theterminal device 7 is being held such that X-axis thereof is oriented inthe gravity direction, that is, in the case where the terminal device 7has been rotated by 90 degrees counterclockwise (about Z-axis) from thereference attitude shown in FIG. 3, when the right analog stick 72B isslid in the Y-axis negative direction (the right direction relative tothe gravity direction), the first virtual camera C1 is rotated abouty-axis. By the first virtual camera C1 being rotated about y-axis, thefirst virtual camera C1 is directed rightward in the virtual space. As aresult, the house object 82, which was only partially displayed in aright part of the screen before the operation of the right analog stick72B, is displayed near the center in the left-right direction of thescreen after the operation.

In the state of the vertical holding shown in FIG. 8, when the rightanalog stick 72B is slid in the Y-axis positive direction (the leftdirection relative to the gravity direction), the first virtual cameraC1 is directed leftward in the virtual space. Further, in the state ofvertical holding shown in FIG. 8, when the X-axis direction (the gravitydirection) is inputted by using the right analog stick 72B, theorientation of the first virtual camera C1 is not changed.

FIG. 9 illustrates a non-limiting example of a change in the orientationof the first virtual camera C1 at the time when a direction input isperformed by using the right analog stick 72B, while the terminal device7 is being held (inclined holding) such that X-axis thereof is inclinedby a predetermined angle relative to the horizontal direction in thereal space. As shown in FIG. 9, when the terminal device 7 is beingheld, rotated in the roll direction (about Z-axis) by a predeterminedangle (for example, 30 degrees) from the reference attitude, the firstvirtual camera C1 is also rotated in the roll direction (aboutZc1-axis). In this state, when the right analog stick 72B is slid to anapparent right direction (lower right direction in the referenceattitude), the first virtual camera C1 is rotated about y-axis in thevirtual space. That is, when the right analog stick 72B is slid to anapparent right direction, the first virtual camera C1 is directedrightward in the virtual space. Accordingly, the house object 82, whichwas displayed in a right part of the screen before the operation of theright analog stick 72B, is moved leftward in the screen after theoperation.

As described above, even when the attitude of the terminal device 7 ischanged, the first virtual camera C1 is controlled so as to be directedtoward a direction in the virtual space that corresponds to an apparentinput direction by the right analog stick 72B. That is, the orientationof the first virtual camera C1 is changed such that the changeddirection of the orientation of the first virtual camera C1 agrees withthe apparent input direction of the analog stick. Here, the “apparentinput direction” means a sliding direction of the right analog stick 72Bbased on the real space. When the terminal device 7 is in the referenceattitude, the apparent input direction coincides with the direction ofthe input vector indicating the input direction by the right analogstick 72B detected by the terminal device 7. On the other hand, when theterminal device 7 is rotated about Z-axis from the reference attitude,the apparent input direction differs from the direction of the inputvector. Specifically, an input vector indicating the input direction bythe right analog stick 72B is rotated in accordance with the attitude ofthe terminal device 7, and the orientation of the first virtual cameraC1 is changed based on the rotated vector.

FIG. 10 shows a non-limiting example of how an input vector inaccordance with the input direction by the right analog stick 72B isrotated in accordance with the attitude of the terminal device 7. Anoperation performed onto the right analog stick 72B of the terminaldevice 7 is calculated as an input vector. As shown in FIG. 10, theinput vector in accordance with the input direction by the right analogstick 72B is expressed as coordinate values in an IxIy coordinatesystem. Ix-axis coincides with the X-axis negative direction for theterminal device 7, and Iy-axis coincides with the Y-axis positivedirection for the terminal device 7. In the case where the terminaldevice 7 is rotated by θr in the roll direction (about Z-axis), when thevalues of the input vector are, for example, (0.707, −0.707), acorrection vector obtained by rotating the input vector by θr iscalculated. Then, the correction vector is used for rotation abouty-axis for the first virtual camera C1. More specifically, for arotation about y-axis for the first virtual camera C1 (rotation in theyaw direction), only an input in the apparent left-right direction usingthe right analog stick 72B is used. Therefore, the first virtual cameraC1 is rotated about y-axis in accordance with the value of the Ix-axiscomponent of the correction vector.

For example, in the case where the rotation angle θr in the rolldirection of the terminal device 7 is 90 degrees (in the case ofvertical holding), when the values of the input vector are (0, −1.0),the correction vector obtained by rotating the input vector by 90degrees is (1, 0). Therefore, in this case, the first virtual camera C1is rotated about y-axis, regarding that the right direction by the rightanalog stick 72B has been inputted. Moreover, in the case where therotation angle θr in the roll direction of the terminal device 7 is 90degrees, for example, when the values of the input vector are (1, 0),the correction vector obtained by rotating the input vector by 90degrees is (0, 1). Therefore, in this case, the first virtual camera C1is not rotated about y-axis, regarding that the up direction by theright analog stick 72B has been inputted.

As described above, a correction vector is calculated by correcting aninput vector indicating an input direction by the right analog stick72B, in accordance with the attitude of the terminal device 7. Then,based on the correction vector, the first virtual camera C1 is rotatedabout the up-down direction axis in the virtual space. Accordingly, evenwhen the attitude of the terminal device 7 is changed, it is possible todirect the first virtual camera C1 in the apparent input direction bythe right analog stick 72B (horizontal direction relative to the gravitydirection).

As described above, in the present embodiment, the orientation of thefirst virtual camera C1 is controlled by an operation of changing theattitude of the terminal device 7 (first operation), and by a directioninput operation using the right analog stick 72B (second operation). Asshown in FIG. 11, by combining the camera control by the first operationand the camera control by the second operation, the orientation of thefirst virtual camera C1 is determined.

(Game Operation Using Terminal Device and Display Device)

Next, a game operation performed by a plurality of people using imagesdisplayed on the terminal device 7 and the display device 2 will bedescribed with reference to FIG. 12 to FIG. 19. First, with reference toFIG. 12 to FIG. 14, images displayed on the terminal device 7 and thedisplay device 2 will be described.

FIG. 12 illustrates a non-limiting example of an operation of rotatingthe terminal device 7 in the yaw direction, and a control of two virtualcameras based on an input in the left-right direction using the rightanalog stick 72B. As described above, the orientation of the firstvirtual camera C1 for generating an image to be displayed on the LCD 71of the terminal device 7 is changed, in accordance with rotation of theterminal device 7 in the yaw direction (about Y-axis) and an input inthe left-right direction by the right analog stick 72B. Similarly to thefirst virtual camera C1, the orientation of the second virtual camera C2for generating an image to be displayed on the display device 2 ischanged, in accordance with rotation of the terminal device 7 in the yawdirection and an input in the left-right direction by the right analogstick 72B. That is, with respect to the yaw direction, the secondvirtual camera C2 operates in conjunction with the first virtual cameraC1. Specifically, the rotation about Yc1-axis in the coordinate systemfixed for the first virtual camera C1 is equal to the rotation aboutYc2-axis in an Xc2Yc2Zc2 coordinate system fixed for the second virtualcamera C2. However, in order to avoid an abrupt change, the secondvirtual camera C2 may be configured to follow the first virtual cameraC1 over a predetermined time period. It should be noted that Xc2-axisextends in the left direction relative to the second virtual camera C2,Yc2-axis extends in the up direction relative to the second virtualcamera C2, and Zc2-axis extends in the imaging direction of the secondvirtual camera C2.

FIG. 13 illustrates a non-limiting example of a control of two virtualcameras based on an operation of rotating the terminal device 7 in thepitch direction. FIG. 14 illustrates a non-limiting example of a controlof two virtual cameras based on an operation of rotating the terminaldevice 7 in the roll direction. As shown in FIG. 13, when the terminaldevice 7 is rotated in the pitch direction (about X-axis), the firstvirtual camera C1 is also rotated in the pitch direction (aboutXc1-axis). On the other hand, even when the terminal device 7 is rotatedin the pitch direction, the second virtual camera C2 is not rotated inthe pitch direction (about Xc2-axis).

Further, as shown in FIG. 14, when the terminal device 7 is rotated inthe roll direction (about Z-axis), the first virtual camera C1 is alsorotated in the roll direction (about Zc1-axis). On the other hand, evenwhen the terminal device 7 is rotated in the roll direction, the secondvirtual camera C2 is not rotated in the roll direction (about Zc2-axis).That is, with respect to the pitch direction and the roll direction, thesecond virtual camera C2 does not operate in conjunction with the firstvirtual camera C1.

As described above, by rotating the terminal device 7 in the yawdirection and/or by sliding the right analog stick 72B in the left-rightdirection, it is possible to cause the first virtual camera C1 tooperate in conjunction with the second virtual camera C2 with respect tothe yaw direction. Moreover, by rotating the terminal device 7 in thepitch direction and/or the roll direction, i.e., by directing the rearsurface of the terminal device 7 in a desired direction upward,downward, leftward, and/or rightward, the first user holding theterminal device 7 can view the desired direction in the up, down, left,and/or right directions in the virtual space. Therefore, the first usercan have a sweeping view of the virtual space in accordance with theoperation performed onto the terminal device 7 held by the first user.

On the other hand, when the terminal device 7 is rotated in the yawdirection, the second virtual camera C2 is also rotated in the yawdirection (about y-axis), but even when the terminal device 7 is rotatedin the pitch direction or the roll direction, the second virtual cameraC2 is not rotated in the pitch direction or the roll direction.Therefore, even if the first user rotates the terminal device 7 in thepitch direction or the roll direction, the range of the virtual spacedisplayed on the display device 2 is not changed. Thus, the image iseasy to see for the second user viewing the display device 2. That is,if an image on the display device 2 is swung upwardly or downwardly orrotated by the first user directing the terminal device 7 upward ordownward or rotating it in the roll direction, the image becomesdifficult to see for the second user. However, in the presentembodiment, since the second virtual camera C2 operates in conjunctionwith the first virtual camera C1 only in the yaw direction, the image iseasy to see for the second user viewing the stationary display device 2.

Next, description will be given of a case where the second userindicates a position on the screen of the display device 2 using thecontroller 5 while images of the virtual space are being displayed onthe terminal device 7 and the display device 2 as described above. FIG.15 shows a non-limiting example of images displayed on the terminaldevice 7 and the display device 2 when a position on the screen of thedisplay device 2 is indicated by using the controller 5.

As shown in FIG. 15, when the second user directs the tip of thecontroller 5 to the display device 2, a pointer 86 which indicates aposition on the screen is displayed on the screen of the display device2. When the pointer 86 is displayed on the display device 2, anindicated object 87 is displayed on the LCD 71 of the terminal device 7.The indicated object 87 is an object having a predetermined threedimensional shape (sphere, for example), and is arranged at a threedimensional position in the virtual space corresponding to the positionon the screen of the display device 2 indicated by the pointer 86. Then,a taken image of the virtual space including the indicated object 87 isdisplayed on the LCD 71 of the terminal device 7. Moreover, an indicator88 indicating the position of the indicated object 87 is displayed onthe terminal device 7. The indicator 88 indicates the positionalrelationship between the user object 81 and the indicated object 87, andis displayed as an arrow extending from the user object 81 toward theindicated object 87, for example. Since the indicator 88 and theindicated object 87 are displayed on the terminal device 7, the firstuser viewing the terminal device 7 can know the position of theindicated object 87, that is, the position indicated by the second user.

FIG. 16 illustrates a non-limiting example of a position at which theindicated object 87 is located. First, the indicated position on thescreen of the display device 2 indicated by using the controller 5 iscalculated. For example, by the imaging section of the controller 5receiving light from the marker device 6 arranged near (e.g., above thescreen) the display device 2, the indicated position (two dimensional)on the screen of the display device 2 is calculated. Next, as shown inFIG. 16, a straight line passing through the indicated position andextending parallel to Zc2-axis for the second virtual camera C2 iscalculated. Then, a point at which the straight line intersects a planewhich is at a predetermined height h from the ground (xy plane) in thevirtual space is calculated as the three dimensional indicated position.Then, the indicated object 87 is located at the three dimensionalindicated position calculated in this manner.

FIG. 17 shows a non-limiting example of how the user object 81 is movedbased on a direction input using the left analog stick 72A of theterminal device 7. As shown in FIG. 17, by inputting a direction usingthe left analog stick 72A of the terminal device 7, the first user canmove the user object from the position 81′ to the position 81. The firstvirtual camera C1 and the second virtual camera C2 also move in thevirtual space in accordance with the movement of the user object 81. Forexample, when an up direction (Y-axis positive direction) is inputted byusing the left analog stick 72A, the user object 81 is moved forward inthe virtual space (i.e., is moved to the depth direction in the screenof the terminal device 7), and the two virtual cameras also moveaccordingly. When a right direction (X-axis negative direction) isinputted by using the left analog stick 72A, the user object 81 is movedin the right direction in the virtual space, and the two virtual camerasalso move accordingly. Further, as shown in FIG. 17, when an upper rightdirection is inputted by using the left analog stick 72A, the userobject 81 is moved to the right forward direction. In this manner, thefirst user can move the user object 81 in any desired direction in thevirtual space, through a direction input using the left analog stick72A.

In the case where the terminal device 7 is being held vertically, when adirection input is performed by using the left analog stick 72A, theuser object 81 is moved in an apparent input direction of the leftanalog stick 72A, as in the case of the operation using the right analogstick 72B. FIG. 18 illustrates a non-limiting example of a movement ofthe user object 81 when a direction input operation is performed byusing the left analog stick 72A while the terminal device 7 is beingheld vertically. As shown in FIG. 18, when the left analog stick 72A isslid in the X-axis negative direction while the terminal device 7 isbeing held vertically, the apparent sliding direction is the updirection in the real space. Therefore, in this case, it is regardedthat an up direction has been inputted by using the left analog stick72A, and the user object 81 is not moved in the right direction but ismoved forward (i.e., the depth direction in the screen) in the virtualspace. Specifically, as in the case of the operation using the rightanalog stick 72B described above, an input vector corresponding to theinput direction by the left analog stick 72A is rotated in accordancewith the rotation about Z-axis for the terminal device 7. Then, themovement direction of the user object 81 is determined in accordancewith the rotated vector.

FIG. 19 shows a non-limiting example of a scene where the user object 81has moved and arrived at the house object 82 indicated by the seconduser. Here, when the user object 81 has arrived at the house object 82,a game A is selected. The house object 82 is an object for selecting agame to be executed, and a game corresponding to the house object 82 isprepared in advance. Further, in the virtual space, a plurality ofselection objects respectively associated with different types of gamesare arranged in addition to the house object 82 which corresponds to thegame A.

FIG. 20 shows a non-limiting example of a scene where the user object 81has moved and arrived at a building object 89 arranged in the virtualspace. As shown in FIG. 20, the building object 89 corresponding to agame B is arranged in the virtual space, and when the user object 81arrives at the building object 89, the game B is selected.

As described above, by the first user operating the user object 81 toselect a selection object associated with each game, the game isexecuted. By indicating a position on the screen of the display device 2using the controller 5 while viewing the image displayed on the displaydevice 2, the second user indicates a selection object (game) which heor she wishes the first user to select. By looking at the indicatedobject 87 displayed on the LCD 71 of the terminal device 7, the firstuser can know the selection object indicated by the second user. Then,the first user operates the user object 81 to select the selectionobject, thereby starting the game.

As described above, the indicated object 87 is located at a position inthe virtual space corresponding to the indicated position on the screenof the display device 2 and the indicated object 87 is displayed on theterminal device 7, whereby communication between the users can beperformed.

It should be noted that the above embodiment is merely an example, andvarious modifications may be performed as described below for example.

For example, in the above embodiment, the first virtual camera C1 isrotated in the yaw direction in accordance with an input in the apparentleft-right direction (left-right direction relative to the gravitydirection), which is performed by using the right analog stick 72B. Inanother embodiment, the first virtual camera C1 may be rotated in thepitch direction, in accordance with an input in the apparent up-downdirection (direction parallel to the gravity). That is, in accordancewith an input in said up-down direction, the first virtual camera C1 maybe directed toward the up-down direction in the virtual space. In thiscase, an input vector indicating the input direction by the right analogstick 72B is corrected (rotated) in accordance with the attitude of theterminal device 7 to calculate a correction vector, and the firstvirtual camera C1 is rotated in the yaw direction in accordance with theleft-right direction component of the correction vector. Further, thefirst virtual camera C1 is rotated in the pitch direction in accordancewith the up-down direction component of the correction vector.

In the above embodiment, a direction input is performed using an analogstick capable of inputting any direction, and the input directionperformed by using the analog stick is corrected in accordance with theattitude of the terminal device 7. In another embodiment, a directioninput may performed using a cross key, a plurality of operation buttonsarranged in a predetermined pattern, or any other input means, and theinput direction may be corrected in accordance with the attitude of theterminal device 7.

In the above embodiment, the second virtual camera C2 operates inconjunction with the first virtual camera C1 only with respect to theyaw direction. However, in another embodiment, the second virtual cameraC2 may operate in conjunction with the first virtual camera C1 also withrespect to the pitch direction and the roll direction. Moreover, inanother embodiment, the second virtual camera C2 may operate inconjunction with the first virtual camera C1 only with respect to apredetermined direction. That is, the first virtual camera C1 may rotatein a plurality of directions (roll, pitch, and yaw) in accordance withan operation onto the terminal device 7, whereas the second virtualcamera C2 may rotate in a predetermined direction among the plurality ofdirections.

In the above embodiment, the first virtual camera C1 and the secondvirtual camera C2 are arranged in the virtual space, and the firstvirtual camera C1 operates in conjunction with the second virtual cameraC2. An image of the virtual space viewed from the first virtual cameraC1 is displayed on the terminal device 7, and an image of the virtualspace viewed from the second virtual camera C2 is displayed on thedisplay device 2. In another embodiment, an image viewed from the firstvirtual camera C1 may be displayed on the terminal device 7 and thedisplay device 2, without setting the second virtual camera C2.

Further, in the above embodiment, the imaging direction of the secondvirtual camera C2 is made parallel (a predetermined angle difference isallowed) to the imaging direction of the first virtual camera C1,whereby an image (image of the virtual space viewed from behind the userobject 81) similar to the image displayed on the terminal device 7 isdisplayed on the display device 2. That is, in the above embodiment, thedirection in the virtual space corresponding to the depth direction forthe display device 2 coincides with the direction in the virtual spacecorresponding to the depth direction for the terminal device 7. However,in another embodiment, the position and the imaging direction of thesecond virtual camera C2 may not coincide with (substantially coincidewith) those of the first virtual camera C1.

FIG. 21 shows non-limiting various examples of the positionalrelationship between the first virtual camera C1 and the second virtualcamera C2 and the imaging directions (sight line directions) thereof. Asin the above embodiment, the second virtual camera C2 may be located ata viewpoint B which (substantially) coincides with a viewpoint A of thefirst virtual camera C1, and an imaging direction B of the secondvirtual camera C2 may (substantially) coincide with an imaging directionA of the first virtual camera C1. Specifically, the viewpoint B of thesecond virtual camera C2 may coincide with the viewpoint A of the firstvirtual camera C1 with respect to the left-right direction and theup-down direction. Alternatively, the viewpoint B of the second virtualcamera C2 may coincide with the viewpoint A of the first virtual cameraC1 with respect to the left-right direction but may not coincide withthat of the first virtual camera C1 with respect to the up-downdirection. In this case, as shown in FIG. 3, substantially the sameimages are displayed on the display device 2 and the terminal device 7,and the direction in the virtual space indicated by the depth directionfor the image on the display device 2 corresponds to the depth directionfor the image displayed on the terminal device 7. In other words, theimaging direction of the first virtual camera C1 and the imagingdirection of the second virtual camera C2 may be set to be substantiallyparallel (closer to parallel than perpendicular) to each other.

Moreover, as shown in FIG. 21, the second virtual camera C2 may belocated at a viewpoint C, and the imaging direction thereof may be setin a direction C. Further, the second virtual camera C2 may be locatedat a viewpoint D, and the imaging direction thereof may be set in adirection D. That is, the imaging direction of the second virtual cameraC2 may be set to be substantially perpendicular (closer to perpendicularthan parallel) to the ground in the virtual space.

FIG. 22 shows a non-limiting example of an image displayed on thedisplay device 2 when the second virtual camera C2 is set at theviewpoint C shown in FIG. 21. FIG. 23 shows a non-limiting example of animage displayed on the display device 2 when the second virtual cameraC2 is set at the viewpoint D shown in FIG. 22. As shown in FIG. 22, inthe display device 2, an image viewed from an upper position in thevirtual space than in the terminal device 7 is displayed. Also in thiscase, since the first virtual camera C1 operates in conjunction with thesecond virtual camera C2, the image displayed on the display device 2changes in accordance with an operation performed onto the terminaldevice 7. Further, as shown in FIG. 23, a bird's-eye view image of thevirtual space viewed from directly above may be displayed on the displaydevice 2. Also in this case, the first virtual camera C1 similarlyoperates in conjunction with the second virtual camera C2. As shown inFIG. 22 and FIG. 23, an image in which the depth direction in the screenextends along the ground in the virtual space is displayed on theterminal device 7 as in the above embodiment, whereas an image in whichthe depth direction in the screen extends (substantially)perpendicularly to the ground in the virtual space is displayed on thedisplay device 2. That is, an image of the virtual space including theuser object 81 viewed from above in virtual space is displayed on thedisplay device 2. In this case, the direction in the virtual spaceindicated by the up direction of the image displayed on the displaydevice 2 corresponds to the depth direction for the image displayed onthe terminal device 7. In other words, the imaging direction of thefirst virtual camera C1 and the imaging direction of the second virtualcamera C2 may be set substantially perpendicularly (closer toperpendicular than parallel) to each other.

In the above embodiment, a position on the screen of the display device2 is indicated by using the controller 5. However, in anotherembodiment, a position on the screen of the display device 2 may beindicated by using any device. For example, a pointer on the screen ofthe display device 2 may be moved by a direction input using: a pointingdevice such as a mouse; an analog stick; a cross key; a plurality ofoperation buttons; and the like, and the position of the pointer may beused as an indicated position. Alternatively, a touch panel may beprovided on the screen of the display device 2, and a position on thescreen of the display device 2 may be indicated by a touch operation.

In the above embodiment, a game corresponding to an object selected by aselection object is executed. In another embodiment, any other programmay be executed corresponding to a selected object.

In the above embodiment, a position on the screen of the display device2 is indicated and the indicated object 87 indicating the indicatedposition is displayed on the terminal device 7. In another embodiment, aposition on the screen of the LCD 71 of the terminal device 7 isindicated, and an indicated object is located at a position in thevirtual space corresponding to the indicated position, whereby theindicated object may be displayed on the display device 2.

For example, a touch panel is provided on the screen of the terminaldevice 7, a position on the LCD 71 is indicated through a touchoperation, whereby an indicated object may be located at a position inthe virtual space corresponding to the indicated position. Then, animage including the indicated object is displayed on the display device2, whereby communication may be performed between the first user usingthe terminal device 7 and the second user viewing the display device 2.Further, a position on the screen of the LCD 71 of the terminal device 7may be indicated by using an operation device other than the terminaldevice 7, and an indicated object may be located at a position in thevirtual space corresponding to the indicated position. Then, an imageincluding the indicated object may be displayed on the display device 2.

(Flow of Processes)

Next, processes performed in the game system 1 of the present embodimentwill be described in detail. First, various types of data stored in thememory will be described. FIG. 24 shows a non-limiting example ofvarious types of data stored in the game apparatus 3.

As shown in FIG. 24, a program 100 for performing later-describedprocesses, a game program 101 for executing the game A, and a gameprogram 102 for executing the game B are stored in the memory 11 and thelike in the game apparatus 3. Attitude data 110 indicating the attitudeof the terminal device 7, input direction data 111 indicating inputdirections of the left analog stick 72A and the right analog stick 72B,first virtual camera data 112 indicating the orientation of the firstvirtual camera C1, second virtual camera data 113 indicating theorientation of the second virtual camera C2, indicated position data 114indicating the position on the screen of the display device 2 indicatedby using the controller 5, object data 115 indicating the position of auser object, and the like are stored in the game apparatus 3. Moreover,data and the like indicating the positions of the house object 82 andthe building object 89 are stored in the game apparatus 3.

(Description of Flowchart)

Next, processes performed in the game apparatus 3 (CPU 10) will bedescribed in detail with reference to FIG. 25. FIG. 25 is a non-limitingexample of a main flowchart showing the flow of the processes performedin the game apparatus 3.

First, the game apparatus 3 obtains (calculates) the attitude of theterminal device 7 based on data which is from the inertial sensor 73 ofthe terminal device 7 and stored in the memory (step S101). Next, basedon operation data which is from the terminal device 7 and stored in thememory, the game apparatus 3 obtains input vectors indicating inputdirections inputted by using the right analog stick 72B and the leftanalog stick 72A, respectively (step S102).

Next, the game apparatus 3 controls the first virtual camera C1 and thesecond virtual camera C2 based on the attitude of the terminal device 7and the input vector of the right analog stick 72B (step S103).Specifically, the game apparatus 3 calculates an orientation of thefirst virtual camera C1 in accordance with the rotation of the terminaldevice 7 in the yaw direction, the pitch direction, and the rolldirection. Moreover, the game apparatus 3 calculates a correction vectorbased on the input vector indicating the input direction by the rightanalog stick 72B and the rotation of the terminal device 7 in the rolldirection, and rotates the first virtual camera C1 about y-axis inaccordance with the calculated correction vector. Moreover, the gameapparatus 3 sets an orientation of the second virtual camera C2 inaccordance with the orientation of the first virtual camera C1 asdescribed above.

Next, the game apparatus 3 obtains an indicated position on the screenof the display device 2 using the controller 5 (step S104), andcalculates a three dimensional position, which is a position in thevirtual space corresponding to the indicated position (step S105). Then,the game apparatus 3 locates the indicated object 87 at the calculatedthree dimensional position (step S106). Through the above processes,images viewed from the first virtual camera C1 and the second virtualcamera C2 are generated respectively, and outputted to the terminaldevice 7 and the display device 2, respectively. Accordingly, an imageof the virtual space including the indicated object 87 viewed from thefirst virtual camera C1 is displayed on the terminal device 7, and animage of the virtual space viewed from the second virtual camera C2which operates in conjunction with the first virtual camera C1 isdisplayed on the display device 2.

Next, the game apparatus 3 moves the user object 81 in the virtualspace, based on the input vector in accordance with the input directionby the left analog stick 72A (step S107). In this case, as describedabove, the input vector is corrected in accordance with the attitude ofthe terminal device 7 and then the user object 81 is moved based on thecorrected vector. Then, it is determined whether the moved user object81 has arrived at the position of the selection object arranged in thevirtual space (step S108). When the user object 81 has arrived at theselection object (step S108: YES), the game apparatus 3 starts executionof a game program stored in the memory that corresponds to the selectionobject (step S109). When the user object 81 has not arrived at theselection object (step S108: NO), the game apparatus 3 ends theprocesses shown in FIG. 25.

It should be noted that the processes in the flowchart shown in FIG. 25are repeatedly performed. The process of each step in the flowchart ismerely an example and the order of processes of the steps may be changedas long as a similar result can be obtained.

Further, a part of the processes described above may be performed in theterminal device 7. For example, in accordance with the attitude of theterminal device 7 and a direction input using an analog stick, theorientation of the first virtual camera C1 may be calculated in theterminal device 7. In this case, the terminal device 7 calculates avector obtained by rotating an input vector of the analog stick inaccordance with the attitude of the terminal device 7, and calculatesthe orientation of the first virtual camera C1 based on the calculatedvector.

Further, the above programs may not be executed in the game apparatus 3,but may be executed in any other information processing apparatus, andthe information processing apparatus may function as the game systemdescribed above. For example, as another information processingapparatus, a mobile phone, a smart phone, a PDA, a personal computer, atablet-type computer, or the like may be used. Such informationprocessing apparatuses may function as the game apparatus 3 and theterminal device 7.

Further in another embodiment, in a game system including a plurality ofapparatuses capable of communicating with each other, the plurality ofapparatuses may share the execution of the above-described gameprocessing performed in the game apparatus 3. For example, a pluralityof information processing apparatuses connected to a network such as theInternet may form the game system as described above.

Further, in the above embodiment, the processes in the above flowchartare performed, by the CPU of the game apparatus 3 executing programs. Inanother embodiment, a part or the whole of the above processes may beperformed by a dedicated circuit included in the game apparatus 3 or bya different general-purpose processor. At least one processor mayfunction as a “programmed logic circuit” for performing theabove-described processes.

The systems, devices and apparatuses described herein may include one ormore processors, which may be located in one place or distributed in avariety of places communicating via one or more networks. Suchprocessor(s) can, for example, use conventional 3D graphicstransformations, virtual camera and other techniques to provideappropriate images for display. By way of example and withoutlimitation, the processors can be any of: a processor that is part of oris a separate component co-located with the stationary display and whichcommunicates remotely (e.g., wirelessly) with the movable display; or aprocessor that is part of or is a separate component co-located with themovable display and communicates remotely (e.g., wirelessly) with thestationary display or associated equipment; or a distributed processingarrangement some of which is contained within the movable displayhousing and some of which is co-located with the stationary display, thedistributed portions communicating together via a connection such as awireless or wired network; or a processor(s) located remotely (e.g., inthe cloud) from both the stationary and movable displays andcommunicating with each of them via one or more network connections; orany combination or variation of the above.

The processors can be implemented using one or more general-purposeprocessors, one or more specialized graphics processors, or combinationsof these. These may be supplemented by specifically-designed ASICs(application specific integrated circuits) and/or logic circuitry. Inthe case of a distributed processor architecture or arrangement,appropriate data exchange and transmission protocols are used to providelow latency and maintain interactivity, as will be understood by thoseskilled in the art.

Similarly, program instructions, data and other information forimplementing the systems and methods described herein may be stored inone or more on-board and/or removable memory devices. Multiple memorydevices may be part of the same device or different devices, which areco-located or remotely located with respect to each other.

While certain example systems, methods, devices and apparatuses havebeen described herein, it is to be understood that the appended claimsare not to be limited to the systems, methods, devices and apparatusesdisclosed, but on the contrary, are intended to cover variousmodifications and equivalent arrangements included within the spirit andscope of the appended claims.

What is claimed is:
 1. A display control system configured to controlhow an image is displayed in accordance with an operation performed viaan individual operation device which includes a direction inputmechanism directly operable by a user, a sensor for detecting change inan attitude of the operation device, and a display, the display controlsystem comprising: an input section configured to obtain input data fromthe direction input mechanism indicative of an input directiondesignated by a user operating the direction input mechanism of theoperation device; a virtual camera first controller configured to changea pointing direction of a first virtual camera in a virtual space basedupon the input data obtained via a user operating the direction inputmechanism, wherein the input data obtained via the direction inputmechanism is modified based upon a detected change in an attitude of theoperation device, and the virtual camera first controller controls thepointing direction of the first virtual camera in accordance with themodified input data; and a display controller configured to cause thedisplay to display an image produced by the first virtual camera.
 2. Thedisplay control system according to claim 1, wherein the virtual camerafirst controller controls the first virtual camera based on an inputdirection designated by a user operating the direction input mechanism,wherein the input direction is based upon a direction designated in realspace.
 3. The display control system according to claim 1, furthercomprising: a virtual camera second controller configured to control anorientation of the first virtual camera, in accordance with the attitudeof the operation device.
 4. The display control system according toclaim 1, wherein the virtual camera first controller changes anorientation of the first virtual camera such that a directional changein the orientation of the virtual camera agrees with a direction in thevirtual space corresponding to an input direction designated in realspace by a user operating the direction input mechanism.
 5. The displaycontrol system according to claim 1, wherein the virtual camera firstcontroller rotates an input direction indicated by the input data inaccordance with a rotation of the operation device about an axisperpendicular to a screen of the display, and controls an orientation ofthe first virtual camera based on the rotated input direction.
 6. Thedisplay control system according to claim 1, wherein based upon adirectional component in a predetermined direction of the inputdirection, the virtual camera first controller changes an orientation ofthe first virtual camera into a predetermined direction in the virtualspace.
 7. The display control system according to claim 3, wherein inaccordance with a rotation of the operation device about an axisperpendicular to a screen of the display section, the virtual camerasecond controller rotates the first virtual camera about an imaging axisthereof in the rotation direction of the operation device.
 8. Thedisplay control system according to claim 1, further comprising: a userobject controller configured to control a moving direction of a userobject movable in the virtual space, based on an input directiondesignated by a user operating the direction input mechanism or a seconddirection input mechanism provided in the operation device, whereininput data indicative of an input direction is further changed inaccordance with a change in the attitude of the operation device, andthe moving direction of the user object is controlled based on thefurther changed input data, and the display controller causes thedisplay to display an image including the user object.
 9. The displaycontrol system according to claim 1, further comprising: a virtualcamera third controller configured to change an orientation of a secondvirtual camera arranged in the virtual space, in accordance with anorientation of the first virtual camera; and a display second controllerconfigured to cause a second display device other than the display ofthe operation device to display an image of the virtual space viewedfrom the second virtual camera.
 10. A display control method implementedusing an information processing system having one or more processors forcontrolling how an image is displayed in accordance with an operationperformed via an individual operation device which includes a directioninput mechanism directly operable by a user, a sensor for detectingchange in an attitude of the operation device, and a display, thedisplay control method comprising: obtaining input data from thedirection input mechanism indicative of an input direction designated bya user operating the direction input mechanism of the operation device;controlling, with the processing system, a virtual camera in a virtualspace and changing a pointing direction of the virtual camera within thevirtual space based upon the input data obtained via a user operatingthe direction input mechanism, wherein the input data obtained via thedirection input mechanism is modified based upon a detected change in anattitude of the operation device, and the pointing direction of thevirtual camera is changed in accordance with the modified input data;and using the processing system to cause the display to display an imageproduced by the first virtual camera.
 11. The display control methodaccording to claim 10, wherein the first virtual camera is controlledbased on an input direction designated by a user operating the directioninput mechanism, wherein the input direction is based upon a directiondesignated in real space.
 12. The display control method according toclaim 10, further comprising: controlling an orientation of the firstvirtual camera, based on the attitude of the operation device.
 13. Thedisplay control method according to claim 10, wherein an orientation ofthe first virtual camera is changed such that a directional change inthe orientation of the virtual camera agrees with a direction in thevirtual space corresponding to an input direction designated in realspace by a user operating the direction input mechanism.
 14. The displaycontrol method according to claim 10, wherein an input directionindicated by the input data is rotated in accordance with a rotation ofthe operation device about an axis perpendicular to a screen of thedisplay, and an orientation of the first virtual camera is controlled inaccordance with the rotated input direction.
 15. The display controlmethod according to claim 10, wherein an orientation of the firstvirtual camera is changed, based upon a directional component in apredetermined direction of the input direction, into a predetermineddirection in the virtual space.
 16. The display control method accordingto claim 12, wherein in accordance with a rotation of the operationdevice about an axis perpendicular to a screen of the display section,the first virtual camera is rotated about an imaging axis thereof in therotation direction of the operation device.
 17. The display controlmethod according to claim 10, further comprising: controlling a movingdirection of a user object movable in the virtual space, based on aninput direction designated by a user operating the direction inputmechanism or a second direction input mechanism provided in theoperation device, wherein input data indicative of an input direction ismodified in accordance with a detected change in the attitude of theoperation device, and the moving direction of the user object iscontrolled based upon the modified input data, and causing the displayto display an image including the user object.
 18. The display controlmethod according to claim 10, further comprising: changing anorientation of a second virtual camera arranged in the virtual space, inaccordance with an orientation of the first virtual camera; and causinga display device other than the display of the operation device todisplay an image of the virtual space viewed from the second virtualcamera.
 19. A non-transitory computer-readable storage medium havingstored thereon a display control program for controlling how an image ofa virtual space is displayed in accordance with an operation performedvia an individual operation device which includes a computer processor,a direction input mechanism directly operable by a user, a sensor fordetecting change in an attitude of the operation device, and a display,the program causing the computer processor to function and performoperations comprising: obtaining input data from the direction inputmechanism indicative of an input direction designated by a useroperating the direction input mechanism of the operation device;controlling a virtual camera in a virtual space and changing a pointingdirection of the virtual camera within the virtual space based upon theinput data obtained via a user operating the direction input mechanism,wherein the input data obtained via the direction input mechanism ismodified based upon a detected change in an attitude of the operationdevice, and the pointing direction of the virtual camera is changed inaccordance with the modified input data; and causing the display todisplay an image produced by the controlled virtual camera.
 20. Adisplay control apparatus configured to control how an image isdisplayed in accordance with an operation performed via an individualoperation device which includes a direction input mechanism directlyoperable by a user, a sensor for detecting change in an attitude of theoperation device, and a display, the display control apparatuscomprising: an input section configured to obtain input data from thedirection input mechanism indicative of an input direction designated bya user operating the direction input mechanism of the operation device;a virtual camera first controller configured to change a pointingdirection of a first virtual camera in a virtual space based upon theinput data obtained via a user operating the direction input mechanism,wherein the input data obtained via the direction input mechanism ismodified based upon a detected change in an attitude of the operationdevice, and the pointing direction of the virtual camera is changed inaccordance with the modified input data; and a display controllerconfigured to cause the display section to display an image produced bythe first virtual camera.
 21. A display control system configured tocontrol how an image is displayed in accordance with an operationperformed via an individual operation device which includes a directioninput mechanism directly operable by a user, a sensor for detectingchange in an attitude of the operation device, and display, the displaycontrol system comprising: an input section configured to obtain inputdata from the direction input mechanism indicative of an input directiondesignated by a user operating the direction input mechanism of theoperation device; a virtual object controller configured to controlmovement and direction of movement of a predetermined virtual object ina virtual space based upon input data obtained via the direction inputmechanism, wherein the input data obtained via the direction inputmechanism is modified based upon a detected change in an attitude of theoperation device; and a display controller configured to display animage of the predetermined virtual object on the display as captured bya virtual camera located in the virtual space and to change anorientation of the virtual camera in accordance with a detected changein attitude of the operation device.